Oscar Kempthorne and the Distribution of Quadratic Forms

For a considerable number of years, Oscar Kempthorne and I were colleagues in the Iowa State University Department of Statistics and shared the responsibility for the teaching of two graduate-level courses on linear models. The first of these was an M.S.-level course in which most, but not all, of the students were statistics majors and for which an introductory course in linear or matrix algebra was a prerequisite. The second was a Ph.D.-level course with an enrollment consisting (aside from a very occasional exception) entirely of statistics majors. Both of these courses included some coverage of the distribution of quadratic forms, as do most courses on linear models. And, as do most instructors of such courses, Oscar and I wrestled with various questions about how this material should be taught. There are two results on the distribution of quadratic forms that are of great importance in linear-model theory. One of them gives a condition that is necessary and sufficient for a quadratic form (in normal variates) to have a (possibly noncentral) chi-square distribution. In some comments published (as a letter to the editor) in the American Statistician (Harville 2000), I offered some opinions on various issues related to how this result should be presented and proved. The second important result on the distribution of quadratic forms gives conditions that are necessary and sufficient for quadratic forms to be distributed independently of each other and of linear forms. It is this second result that is the subject in what follows. Let x represent an random vector whose distribution is MVN (multivariate normal) with mean vector and variance-covariance matrix V. And, let A and B represent symmetric matrices, and take a and b to be -dimensional column vectors. Further, define a x x Ax and b x x Bx (where and are scalars). In the important special case where V is nonsingular, a necessary and sufficient condition for two linear forms a x and b x to be statistically independent is that a Vb , a necessary and sufficient condition for a linear form b x and a quadratic form x Ax to be statistically independent is AVb 0 (or equivalently b VA 0 ), and a necessary and sufficient condition for two quadratic forms x Ax and x Bx to be statistically independent is AVB 0. For purposes of both proof and presentation, I find it convenient to regard all three of these results as special cases of the following theorem. Theorem 1. Suppose that V is nonsingular. Then, for and to be statistically independent, it is necessary and sufficient that